Investigation on application of CLIP model for multimodal dataset

 


   Note: An Official Proposal has not been Submitted to NACC.


   Lead Investigator:    Zilun Zhang
   Institution      :    University of Toronto
   E-Mail           :    zilun.zhang@mail.utoronto.ca
   Proposal ID      :    1674



   Proposal Description:
   The project is about how multimodal data can improve the diagnosis result comparing with just image data. We plan to apply the idea of CLIP model (openai) first on the multimodal NACC dataset first as a baseline for the multimodal model. After that, we plan to come up with a new model for the multimodal dataset with a patient-wise data augmentation method for the multimodal dataset.